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Abstract. In this paper the characterization as convolution operators of fil¬ 
ters sending finite energy signals to bounded signals is used to prove several 
theoretical results concerning the distance between the ideal filter and the 
spaces of physically realizable filters. Both the analog and the digital cases are 
studied and the formulas for the distance and the angle between the filters in 
each case are also given. 


1. Introduction and motivation. One of the main principles on which the deter¬ 
ministic mathematical physics is based is the statement that Nature obeys certain 
universal laws, and these laws can be usually expressed in terms of partial differen¬ 
tial equations. Moreover, this view is commonly extended assuming some additional 
properties. For example, there is a strong feeling supporting that Nature is uniform. 
This means that a law that holds true at a certain place and a certain moment holds 
true everywhere and all times. In particular, this implies that the involved partial 
differential equations in the description of the physical laws should have coefficients 
independent on the time. Moreover, under certain additional conditions, many of 
these laws are linear in nature. Hence, the continuous linear operators defined be¬ 
tween two function spaces that have the property of being translation-invariant are 
of great interest in Applied Mathematics. These operators are usually named filters 
by the engineering community (See [TJ p. 14 and Definition 34.1.1]). 

More precisely, let X and Y be two function spaces with the property that 
their elements are functions of the real variables ti,... ,tn and, for each h G K”, 
let Th : K" — 7- R" be the translation operator Tfi(a) = a — h. By definition, 
an analog filter with input signals from X and output signals from T is a con¬ 
tinuous linear operator L : X —>■ Y such that, for every h G MY, the relation 
L {x{Th{s))) {t) = L{x{s)){Th{t)) holds true. On the other hand, if X,Y are se¬ 
quence spaces (i.e. their elements are n-dimensional sequences {ofelfegz"), we say 
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that L : X —> F is a digital filter if it is a continuous linear operator and the rela¬ 
tion L{{xk}k&^) = {i/fclfeez" implies the relation L {{xk-N}k&^) = {yk-N}k&^ 
for every € Z". The best known examples of filters are the convolution operators, 
which are operators of the form 

/ OO 

x{s)h{t — s)ds. 

-OO 

Indeed, the name “filter” has its origin in the fact that under quite general conditions 
the Fourier transform of a convolution is a product, so that the filters Lh can 
be represented in the frequency domain as F = X ■ H, where X = X{x), H = 
J-{h) and F = F{L(x)). Hence, if the function H{^) satisfies H{^) = 1 for the 
frequencies ^ G Wi and H{^) = 0 for ^ G W 2 then we understand that Lh allows 
the frequency content of the signal x that belongs to the set Wi C R but removes 
the frequency content belonging to W 2 C R, so that we may interpret that Lh filters 
some frequency contents of x. 

Filter Theory is an interesting branch of Mathematical Analysis that contains 
many beautiful results. For example, the characterization of filters between some 
kinds of function spaces X, Y strongly depends on the geometric and/or analytical 
properties of these spaces, and it is usually a difficult problem (See [3] for a classical 
paper, [dj Chapter zero] for an historical overview and [5],i for some recent results. 
For distributions, a classical result by L. Schwartz stating that every filter T : 
£' —>■ 2?' is a convolution operator (where £' denotes the space of distributions 
with compact support and £>' is the space of distributions), is to be found in [141 
Theorem 5.8.1, page 144]). Moreover, this theory is clearly related to such diverse 
subjects as Fourier Analysis, Functional Analysis and Signal Processing. From 
now on, we will restrict our attention to the one-dimensional case (n = 1) because 
it is precisely in that context where physically realizable and causal filters have 
are meaningful. Under this assumption, it is quite natural to identify our unique 
independent variable, t, with time. 

Let us accord that a filter L is physically realizable if there exists a certain 
T > 0 such that for every time value t G R the output of the filter at t depends 
on the values of the input signal on the interval (—oo,t -|- Tj. This means that 
(at least theoretically) the filter can be realized under the understanding that we 
allow some delay time T. The filter does not depend on all the past and future 
values of the signal but just on the past and perhaps some future values limited in 
time. When T = 0 we say that the filter is causal. The concept of causal filter was 
introduced in the literature by N. Wiener in 1926 in a paper devoted to give an 
adequate interpretation of Heaviside’s operational calculus That paper is also 
famous because it was an inspiration point for the introduction by L. Schwartz of 
his theory of distributions. 

Now, it is well known that the analog ideal filter, which is given in the frequency 
domain by 

L{Xm=X{0H{0, (1) 

where H = and X[,,,t](0 = 1 for S [a, 6], X[„,i,j(0 = 0 for ^ ^ [a,b], is not 
physically realizable. The simplest reason is that the inverse Fourier transform of 
X[„ does not vanish over any interval of the form (— 00 , — T). Another explanation 
appears when we take into account an important theorem by Paley and Wiener [5J 
Theorem XH, p. 16] (9] p. 35] where they characterize physically realizable filters 
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in the frequency domain as those of the form ([T]) for which G L^(K) and 

I log 1^(01 I .. . 

(With such a characterization it is obvious that i?(^) cannot vanish on any open set 
of the real line!). Wiener was so proud of this characterization that he commented 
the result several times in printed form. For example, in m p. 37] he said: 

This [result] plays a very important part in the theory of filters. It states that, in any realizable 
network whatever, the attenuation, taken as a function of the frequency tv, and divided by l + tv^, 
yields an absolutely integrable function of the frequency. This results from the fact that the 
attenuation is the logarithm of the absolute value of the transform of [the response to the unit 
impulse] fit) which vanishes for negative t; or, in other words, because strictly no network can 
foretell the future. Thus no filter can have infinite attenuation in any finite band. The perfect 
filter is physically unrealizable by its very nature, not merely because of the paucity of means at 
our disposal. No instrument acting solely on the past has a sufficiently sharp discrimination to 
separate one frequency from another with unfailing accuracy. 

Moreover, in his autobiography [131 p. 168], when speaking about his mathe¬ 
matical work with Paley, he said: 

One interesting problem which we attacked together was that of the conditions restricting the 
Fourier transform of a function vanishing on the half line. This is a sound mathematical problem 
on its own merits, and Paley attacked it with vigor, but what helped me and did not help Paley 
was that it is essentially a problem in electrical engineering. It had been known for many years 
that there is a certain limitation on the sharpness with which an electric wave filter cuts a fre¬ 
quency band off, but the physicists and engineers had been quite unaware of the deep mathematical 
grounds for these limitation. In solving what was for Paley a beautiful and difficult chess problem, 
completely contained within itself, I showed at the same time that the limitations under which the 
electrical engineers were working were precisely those which prevent the future from influencing 
the past. 

The result was, moreover, a key step for the proof of several fundamental the¬ 
orems in Harmonic Analysis such as Carleman’s characterization of quasi-analytic 
functions. In our opinion, Wiener was in his own right to be proud of his result and 
all the “philosophical” interpretations he gives to it are essentially correct and illu¬ 
minating. We have wondered if a certain quantitative estimation of the “far away” 
ideal filters are of being physically realizable already exists. We have not found 
such an estimation in the literature. The main goal of this paper is to make several 
computations in this direction. In order to give a precise focus to our computations 
we first introduce a result which characterizes certain hlters as convolution filters. 
After that, we dedicate a section to the analog case and another to the digital one. 

2. Characterization of filters as convolution operators. Let Ba = B(L^(R), 
L°°{W)) be the normed space of linear bounded operators L : L°°{M.), 

and let us consider Fa C Ba be the subspace of analog hlters. Thus, L belongs to Fa 
if it sends hnite energy analog signals to bounded analog signals, is a bounded linear 
operator and is time invariant. Analogously, we can dehne Bd = B(£^(Z), £°“(Z)), 
the normed space of linear bounded operators L : £^(Z) —^ f°°(Z), and Fd C Bd, 
the subspace of digital hlters. Thus, L belongs to Fd if it sends hnite energy digital 
signals to bounded digital signals, is a bounded linear operator and is time invariant. 
We devote this section to characterize the elements of Fa and Fd as convolution 
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operators in such a way that both spaces are naturally doted of a Hilbert space 
structure. These results are not new (see, for example, m, cni, where several 
generalizations of them are proved). By the contrary, they are well know but they 
are also not attributable to any person. Moreover, they are not available in the 
literature in the simple version we have stated here but they are just special cases 
of quite difficult results. Thus we think a direct proof will make them more visible 
and useful. 


Theorem 1. The map (j) : Fa given by ((iQi) = Lh, where Lh{x) = x*h, 

is an isometry. In particular, Fa, with its usual norm (inherited from Baj is a 
Hilbert space with inner product given by {Lh,Lg) = {h,g). 


Proof. Obviously, <)> is a linear injective map but, in principle, there is the possi¬ 
bility that some elements L of Fa are not of the form Lh, for a certain h G L^(R). 
Thus, we must prove that (j) is surjective and preserves the norm. Given L G Fa 
and t G R, the map Lt : T^(R) — C, defined by Lt{x) = L{x){t), is a linear 
bounded functional. It follows from the Riesz representation theorem la p. 24] 
that there is yt G T^(R) such that Lt{x) = {x,yt), for any x G L^(R). Let us 
denote y(t, s) = yt{s). Time invariance of L means that for every K gR and every 
X G L^(M), 

L (x(- + K)) (t) = L{x){t + K). 

Hence 

{x{- +K),yt) = {x,yt+K), 

so that 

/ OO p OO p oo 

x{s + K) y(t, s) ds = / x{t) y(t,T — K) dr = / x{s) y(t + K, s) ds. 

-OO —OO J —OO 


This implies that yit, s — K) = y(t + K,s). If we take h(t) = y[t, 0), then 


y{t, s) = y{t, 0 - (-s)) = y{t - s, 0) = h{t - s). 

Hence L = Lh- Let us now compute the norm oi L = Lh- 

By definition, ||L|| = sup ||La;||^oo. Now, for each x G L‘^{R) we have that 


\Lx{t)\ = 


/ OO 

a;(s) h{t — s) ds 

-OO 


<lkL. \\Ht--)L. = \\xi 


so that ||L|| < ll/iLa- On the other hand, taking x(f) = h{—t) we have that 


|La;(0)| = 

and therefore 


/ OO p CO 

h{—s)h{—s)ds — / \h{ 

-OO J —OO 


ll^ll > \\L 


11,2 


> 


s)r ds = 


\Lx{0)\ 


11,2 


= X 


11,2 ■ 


L2 I 


Hence ||L|| = ||^11^.2- This proves the theorem. 

The following theorem can be proved with the very same technique. 


□ 


Theorem 2. The map g) : —> Fj given by (p{h) = Lh, where Lh{x) = x * h, 

is an isometry. In particular, Fd, with its usual norm (inherited from Bdj is a 
Hilbert space with inner product given by {Lh, Lg) = {h,g). 
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These two results allow us a perfect and natural identification between filters 
sending finite energy signals to bounded signals and the functions defining them as 
convolution operators. In particular, it is possible to compute the distance between 
two given filters (in the sense of their norms) as the distance between their unit 
impulse responses. Moreover, the computations can be made both in the time 
domain and in the frequency domain since the Fourier transform preserves norms 
(except for a multiplicative constant). 

As we have already said, in this paper we will pay special attention to physically 
realizable filters and causal filters. Thanks to the previous theorems, these filters 
are easily characterized in the time domain. Concretely, the filter L G Fa given by 

L = Lfi is causal (a physically realizable filter with time delay T > 0) if and only if 

^l(-oo 0 ) “ ® -T) = respectively). Analogously, the filter L G Fd given by 

L = Lh is causal (a physically realizable filter with time delay A^ > 0) if and only if 

we have that h[n] = 0 for all n G Z with n < 0 (n < —N, respectively). We denote 
these classes of filters as CFa, CFd, and DFjv, respectively. 

3. The analog case. In this section we will use the notation T[Q_h] for the ana¬ 
log filter with transfer function ^. We would like to estimate the distance 
c^(A[a,h], CFa) and the angle determined by T[a,6] and CFa. 

We need first to recall some concepts and notations. The Fourier transform is 
the operator T : L^(]R) —L^(K) given by 



The inverse Fourier transform is 



These operators are isometries. 

Let us start by computing the function h = b]) (obviously, L[a 5 ](x) = 

X * h). 





h{t) 



Clearly, the function r(t) = |h(t)p = [1 — cos((6 — a)t)] satisfies T{t) = 

T{—t). Let us define 


h_{t) = h(t)X(_^ „)(t) and h+{t) = h{t) 
Then L[a,f)] = L/i = Lh_ + G CFa and 
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Hence Lt^ is the best approximation to the ideal filter L\^a,b] by elements of CFa 
and the corresponding distance is given by: 


Cf = d(L[,,,],Cra)2=||L,J|2=|l/i_f 



Moreover, now is obvious that the the angle determined by and CFa is 


9 = arcsin = arcsin —= — 


TiTiT — ■= — —. 

I|/i|| 72 4 


Thus we have proved the following result: 

Theorem 3. For every frequency interval [a, 6] C M, the ideal analog filter T[a,b] 
forms an angle with the space of analog causal filters CFa which is equal to 7r/4. 
Moreover (and in accordance with this phenomenon) the distance d = CFa) 



Note that the angle does not depend on the frequencies interval [a, b] and the 
distance depends just on its size. One may wonder why this is so. The natural 
answer is that the frequency response H{f) of any ideal hlter satisfies iJ(R) C R, 
so that it satisfies the assumption of the following general theorem: 

Theorem 4. Let us assume that iJ(R) C R, where FI is the transfer function of 


the analog filter L : L^(R) —> L°°(R). Then (i(L, CFa) = '^1711 ^^6 angle 

determined by L and CFa is 7r/4. 

Proof. Set h = T~^{Fl). Then 



and 



satisfy A{—t) = A{t) and B{—t) = —B{t), so that ip{t) = |/i(t)P is an even function 
and we can apply to L the same decomposition technique we used for the ideal filter 


□ 



Let us now compute the distance of (and the angle determined by) the ideal filter 
Lia,b] to the space of physically realizable filters after a delay time T > 0. These 
numbers can be computed by the same procedure, the main idea being to use the 
decomposition ft, = where 


g-{t) = h{t)X(_^,_T)i.t) and g+{t) = h{t) 


The distance is now given by 




Of course, we have that 
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Hence 

d{T) = ( ^ - i ^ [1 - cos((5 - a)t)] dty. 

Now it is not difficult to prove the following result: 

Theorem 5. Let d{T) and 0(T) denote the distance and the angle determined by 
L]^a,b\ 0 ,'^d Dt, respectively. Then 

(a) For every fixed interval [a, b] and every T > 0, 9{T) € [0, 

(b) For any fixed interval [a, b], lim^_^_^ difF) = lim^^^ 0{T) = 0, 

(c) For any fixed T > 0, there exists a constant c{T) € [0, such that linr((^_^j^^ d{T) 
ciT). Moreover, lim(t_^)_^^ 0(T) = 0, 

(d) For any fixed T > 0, d{T) = 0 and difT) = f • 

Note that if we allow any positive delay time T > 0, the angle determined by the 
ideal filters with a large bandpass interval is almost zero, what is in clear contrast 
with the result for causal filters (i.e., for T = 0). 

A philosophical remark One may wonder why the angles we have got belong 
to the interval [0,7r/4]. For example, why ideal filters are never orthogonal to the 
causal ones? The natural answer is: The filter Lh is orthogonal to CF^ if and only 
if it is a filter without memory! (Reason: For the filter Lh, being without memory 
means that h^^^ = 0, which is equivalent to claim that h _L L^(0, oo)). Thus, in a 

“philosophical” sense, the angle between L and CF^ is zero if and only if the filter 
is causal, the angle is 7r/2 if and only if the filter has no memory and the angle 
7r/4 may be related to the fact that the filter uses all values (past and future) of 
the input signal. Of course, this is the case for the ideal filters and, in particular, 
we have shown that if we want to isolate exactly one frequency component of the 
signal, the associated filter determines an angle of 7r/4 with for every T > 0. 

4. The digital case. Let us consider a digital filter L : £^(Z) —> £°°(Z). Then 
Lx = x*hioi a certain sequence h = G £^(Z). In this case, the discrete 

Fourier transform P is given by 

OO 

= E x[n]e-^^^ = Xiw), 

n——co 

and satisfies 

MV 7100 / "L2(0,27r) " " £2 

Moreover, 

Fd{x * h){w) = Xd{x){'w) Fd{h){w) = X{w)H{'w), 

so that we can think about the ideal filter with bandpass given by [a, &], 0 < a < 
b < 27r, as the digital filter L[a,b]X = x * h[a,b] with transfer function H[a,b](,w) = 
J'dih[a,b])iw) = X[„ 6](?'?)• Unfortunately, this filter is not physically realizable. 

Digital causal filters L : £'^{Z) — > £°°{Z) are characterized in the frequency 
domain precisely as those with transfer function a 27r-periodic function H(w) G 
CF(i := span{e“*“'*^ : fc G N} C L^(0, 27r) (here span denotes the closure of the 
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subspace generated by {e € N} in i^(0, 27r)). It follows from Theorem [2] 

and equation @ that 

diL[a,b]^CFd) = d(x[a.&], CFd). 

V 27r 

Obviously, X[o, b] ^ T^(0,27r) so that, denoting by {ck} its Fourier coefficients, 
X[a b] (^) = J2T=-oo Moreover, H{w) = J2T=o is the best approx¬ 

imation of X[a, 6] (^) in fii6 L^-norm by elements of CF^. Therefore, 

OO 

■= d{Xi^,b]^'^df = \\X[a,b] - H{w) f = 271 ^ |cfep. 

k=l 


Now, 


since \ck\ 


b-a= IIXu.mIP = 27r|coP + 2^2 


C-k\ because X[a 6](i^) — Moreover, |coP = -^{h— a)^, and hence 


d{L[a,b],CFd) = 




b — a 1 b — a 

27r V 2 ~ 471 ’ 


( 3 ) 


Thus, we have proved: 


9 


= arcsin 



b — a 
47r 


( 4 ) 


Theorem 6. The angle 0 determined by the ideal filter T[a,b] o,nd the space of digital 
causal filters CF^, and the distance d(L[a,b], CF^) are given by formulas o and 
respectively. In particular, they are both functions of the length of the bandpass 
interval [a,b]. Moreover they satisfy the following properties: 

(a) 0 < 6» < 7r/4, lim(,_^,^„ 9 = 7r/4, lim(,_^,^^^ 9 = Q, 

(b) lim(,__^,^„ CFd) = lini(,_^,^2, '^(■^[a.b], CF^) = 0. 

Note that the distance and the angle do not depend on the location of the fre¬ 
quencies interval [a, b] but just on its size. Moreover, the angle is never greater than 
7r/4. 


Remark If L G Fd has a transfer function H{w) with the property that H{M.) C R, 
then ||R|p = 27r|co(R)P -I- 2d{H,¥d)^■ Hence, if |co(R)| = n certain 

constant C, the angle determined by L and the space CFd is 


9 = arcsin 


l-C'2 


In particular, if H is an odd function then 9 = 7r/4. 


Let us now compute the distance of (and the angle formed by) the ideal hlter 
Lla,b] to the space DF^v of physically realizable (after a delay time N > 0) digital 
filters. First of all, we define 


©Fat = J^{{h G f{Z) : U G DFjv}). 












HOW DISTANT IS THE IDEAL FILTER OF BEING A CAUSAL ONE? 


9 


Clearly, d(x ,DFAr)2 = J2T=n+i |cfc(Xu,6i)P =: d{Nf. Now, 


N 


IIX[a,6]f = b-a = 2Tr = ‘2 d{Nf + 2 tt 


k——c 


k=-N 


N 


= 2d(iV)2 + 27r|co(X[„,„])P + 2^27r|cfe(X(„„)|^ 


fc=i 


We know that 27r |co(x[o.&])P = {b— a)"^ 1271. Moreover, an easy computation shows 
that, for fc ^ 0, 27r |c/c(X[a,i,])P = ~ cos{k{b — a))) so that 




d{N) jb — a j 1 b — a ^ 1 — cos{k{b — a)) 
2 477 k^ 7 r(b—a) 


and the angle determined by i[a,b] and DF^r is 


N 


, , il b-a 1 - cos(A:(& - a)) 

0(iV)=arcsm 


2 47r k'^Tiih — a) 


, ( 5 ) 


( 6 ) 


Thus the following result holds true: 


Theorem 7. The angle 0{N) determined by the ideal filter T[a,t)] and the space 
DF 77 ; and the distance , DF 77 ^ are given by Jormulas (I0p and © , respec¬ 

tively. In particular, they are both functions of the length of the bandpass interval 

[a, &1. Moreover they satisfy the following properties: 

' (aj 0 < e{N) < Trh 0{Nf= ^f4, lim^,_„,^,^ B{N) = 0, 

(b) , DFtv) a,)-j-27r ^^at) 0? 

(c) Given the bandpass interval [a,b] C (0,27r); the sequence is strietly 

decreasing and — 0- 
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